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Abstract 

Given a machine U, a c-short program for x is a string p such that U (p) = x and the length of p 
is bounded by c + (the length of a shortest program for x). We show that for any universal machine, 
it is possible to compute in polynomial time on input x a list of polynomial size guaranteed to 
. contain a <9(log [x[)-short program for x. We also show that there exist computable functions that 

map every x to a list of size <9(|x| 2 ) containing a <9(l)-short program for x and this is essentially 
optimal because we prove that such a list must have size £2(|x| 2 ). Finally we show that for some 
machines, computable lists containing a shortest program must have length £2(2^). 
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^ ■ 1 Introduction 

; The Kolmogorov complexity of a string x is the length of the shortest program computing it. Determin- 

ing the Kolmogorov complexity of a string is a canonical example of a function that is not computable. 
Closely related is the problem of actually producing a shortest program for x. This problem is also 
not algorithmically solvable. When faced with a function that is not computable, it is natural to ask 
C""- whether it can be effectively approximated in a meaningful way. This question has been investigated for 

Kolmogorov complexity in various ways. First of all, it is well-known that the Kolmogorov complexity 
can be effectively approximated from above. A different type of approximation is given by what is typ- 
ically called list computability in algorithms and complexity theory and traceability in computability 
theory. For this type of approximation, one would like to compute a list of "suspects" for the result of 
the function with the guarantee that the actual result is in the list. Of course, the shorter the list is, the 
better is the approximation. 

The list approximability of the Kolmogorov complexity, C(x), has been studied by Beigel et 
al. llBBF+061 . They observe that C(x) can be approximated by a list of size (n — a) for every constant 
a, where n = \x\. On the other hand, they show that, for every universal machine U, there is a constant 
c such that for infinitely many strings x (in fact for at least one x at each sufficiently large length n), 
any computable list containing Cu{x) must have size larger than n/c. 

In this paper we study list approximability for the problem of producing short programs. In order 
to describe our results, we need several formal definitions. 
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A machine U is optimal if Cy(x\y) < Cv{x\y) + 0(1) for all machines V (where the constant 0(1) 
may depend on V). An optimal machine U is standard^, if for every machine V there is a total com- 
putable function t such that for all p,y: U(t(p),y) = V(p,y) and \t(p)\ = \p\ + 0(1). For results that 
hold in polynomial time, we additionally assume these functions t run in time polynomial in \p\. Let 
U (p) stand for U (p,the empty string) and Q/(x) for Q/(xjfhe empty string). A c-short program for x 
with respect to U is a string p that satisfies U(p) =x and \p\ < Cu(x) + c. 

Given an optimal machine U, a list-approximator for c-short programs is a function / that on every 
input x outputs a finite list of strings such that at least one of the elements in the list is a c-short program 
for x on U. Let |/(x)| denote the number of elements in the list f(x). Obviously, for every optimal U, 
there is a (trivial) computable list-approximator / such that \ f(x)\ < 2^ +0 ^\ 

The question we study is how small can \f(x) | be for computable list-approximators / for c-short 
programs, where c is a constant or 0(log |x|). At first glance it seems that in both cases \ f(x) \ must be 
exponential in \x\. Surprisingly, this is not the case. We prove that there is a computable approximator 
with list of size 0(\x\ ) for c-short programs for some constant c depending on the choice of the 
standard machine U. And we show that this bound is tight. We show also that there is a polynomial 
time computable approximator with list of size poly(|x|) for c-short programs for c = 0(log |x|). 

We start with the positive results, i.e., the upper bounds. We show that for every standard machine, 
there exists a list-approximator for 0(l)-short programs, with lists of quadratic size. 

Theorem 1.1. For every standard machine U there exists a computable function f that for any x 
produces a list with 0(\x\ 2 ) many elements containing a program pforx of length \p\ = Cu(x) + 0(1). 

If we allow 0(log |x|)-short programs we can construct lists of polynomial size in polynomial time. 

Theorem 1.2. For every standard machine U, there exists a polynomial-time computable function 
f that for any x produces a list with poly(\x\) many elements containing a program for x of length 
Cu(x) + 0(log |*|)| 

Now we move to the lower bounds. We show that the quadratic lower bound in Theorem 11.11 is 
optimal: it is not possible to compute lists of subquadratic size that contain a 0(l)-short program. 

Theorem 1.3. For all c > 0, for every optimal U, for every computable f that is a list-approximator 
for c-short programs, 

\f(x)\>a(\x\ 2 /c 2 ), 

for infinitely many x. (The constant hidden in ^.-notation depends on the function f and machine U.) 

A weaker linear lower bound can be easily derived from a result of Bauwens MBaul2ll . improving 
a theorem of Gacs HGac74i The result states that Cu(Cu(x) \ x) is greater than log |x| — 0(1) for 
infinitely many x. Thus, for infinitely many x, 

log |x|- 0(1) <Cu(Cu(x) | *)< log |/(jc)|+21ogc + 0(l), 

and therefore \f(x) \ >Q.(\x\/c 2 ). 

The next theorem shows that, at least for some standard machine U, we can not compute lists of 
subexponential size containing a program of length exactly C (x). 

1 This notion was introduced by Schnorr IISch751 . and he called such machines optimal Godel numberings (of the family 
of all computable functions from strings to strings). We use a different term to distinguish between optimal functions in 
Kolmogorov's sense and Schnorr's sense 

2 Recently, Jason Teutsch |Teul2| has improved this result replacing 0(log |x|) by 0(1). 
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Theorem 1.4. There exists a standard machine U such that for every computable f that is a list- 
approximator for 0-short programs, 

\f(x)\>2^-°( l \ 

for infinitely many x. (The constant hidden in O-notation depends on the function f. J^| 

The construction of U can be easily adapted to show that for every c there is a standard machine 
U c satisfying the theorem for c-short programs. This exponential lower bound does not hold for all 
standard machines (see Remark[T]below). 

A few words about the proof technique and paper organization. Essentially, list-approximability 
reduces to a weak form of (polynomial-time) computable compression to (close to) minimal length, 
such that decompression is computable. Inspired by HMRSlll . we achieve such compression by 
using unbalanced bipartite graphs which allow on-line matching (one receives requests to match left 
nodes and each request has to be satisfied before seeing the next one). The connection between 
list-approximability and graphs with on-line matching is studied in Section |2 In the proofs of upper 
bounds we use non-explicit and explicit bipartite graphs of small degree that have expander-like 
properties and hence have good matching abilities. The lower bounds are based on lower bounds for 
the left degree of such graphs. The upper bounds, i.e., Theorem II. II and Theorem 11.21 are proved in 
Section[3] The lower bounds, i.e., Theorem 1 1.3 l and Theorem ll.4l are proved in SectionH] In Section|5] 
we observe that our construction can be used to improve Muchnik's Theorem HMuc02|[MRSlll . and a 
result concerning distinguishing complexity HBFL01II . 



2 List approximators for short programs and on-line matching 

We show that the problem of constructing approximators for short programs is equivalent to construct- 
ing families of bipartite graphs of a certain type. Let a bipartite graph G = (L,R,E C L x R) be given, 
where the set L of left nodes and the set R of right nodes consist of binary strings. Assume that we 
receive "requests for matching" in the graph, each request having the form (a binary string x € L, a 
natural number k). Such a request means that we have to provide to the left node x a match of length at 
most k or slightly more. It might happen that we receive a request (x,k) and later a request (x,k') with 
the same x, but another k' < k. In this case we are allowed to match x to a second right node, thus x has 
now two matching right nodes. Assignments cannot be changed and different right nodes must have 
different matches (however any left node x may have different matches). We will sometimes call right 
nodes hash-values. 

Definition 2.1. Let c(n) be a function of n with natural values. A bipartite graph whose left and right 
nodes are binary strings has matching with overhead c(n) if the following holds. For every set S of pairs 
(x € L,k) having at most 2 k pairs with the second component k (for all k) one can choose for every pair 
(x,k) in S a neighbor p(x,k) ofxsofhat |/?(jc,&)| <&+c(|jc|) and p{x\,k\) ^ pfafa) wheneverxi 7^x2- 
If this is done, we will say that p(x,k) matches x. 

A bipartite graph has on-line matching with overhead c(n) if this can be done in the on-line fashion: 
requests for matching (x,k) appear one by one and we have to find p(x,k) before the next request 
appears. All the made assignments cannot be changed. 

3 The construction implies the existence of such U with a stronger universality property: for every machine V there exists 
a string Wy such that U(wvp\z) = V(p\z) for all p,z- 

4 Independently, this result was obtained by Frank Stephan (personal communication). 
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A bipartite graph is (polynomial time) computable if given a left node x we can compute (in poly- 
nomial time) the list of all its neighbors. 

Theorem 2.2. Assume there is a computable graph with L = {0, 1}* where the left node x has degree 
D(x) and which has on-line matching with overhead c(n). Assume further that the matching strategy 
is computable. Then for every standard machine U there exists a computable function f that for any x 
produces a list with D(x) many elements containing a program p for x of length \p\ = Cu(x) +c(\x\) + 
0(1). If the graph is polynomial time computable then the function f is polynomial time computable, 
too. 

Proof. Run the optimal machine U (q) in parallel for all strings q. Once U (q) halts with the result x we 
pass the request (x, \q\) to the matching algorithm in the graph and find a hash value p of length at most 

\q\ +c(|jc|) for x. 

By construction, every string x is matched to a string p of length at most Cu(x) +c(\x\). Each right 
node is matched to at most one node in the graph. Hence there is a machine V such that V(p) = x 
whenever p is matched to x. Thus for every string x there is a neighbor p of x with \p\ < C\j (x) + c( \x\ ) 
and V(p) = x. As U is a standard machine, there is a (polynomial time) computable function t with 
U(t(p)) = V(p) and \t(p) \ < \p\ +0(1). Let f(x) be the list consisting of t(p) for all the neighbors p 
of x in the graph. By construction \ f{x) \ = D(x) and we are done. □ 

Remark 1. If c(n) = c is a constant function, then we can construct a standard machine U\ which has a 
computable approximator for 0-short programs with lists of size D(x). To this end let U\(0p) = V(p) 
and U\ (l c+2 p) = U (p) and let t (p) = Op in the construction of / in the proof of the above theorem. By 
Theorem 12.4 I below there is a graph having on-line matching with constant overhead and degree 0(|x| 2 ). 
Thus there is a standard machine U\ which has a computable approximator for O-short programs with 
lists of size 0(|x| 2 ). 

There is also a reduction in the other direction. 

Theorem 2.3. Assume that c(n) is computable function and there are an optimal machine U and a 
computable function f that for any x produces a finite list containing a program p for x of length 
\p\ < Cu{x) +c(\x\). Consider the bipartite graph G with L = {0, 1}* where the neighbors of node x 
are all strings from f(x). Then G has on-line matching with overhead c(\x\) + 0(1). 

Proof. For each n let G„ be the subgraph of G with L = {0, 1}-". W.l.o.g. we assume that all strings 
in f(x) have length at most \x\ + 0(1) and hence the graph G n is finite. We claim that G n has on-line 
matching with overhead c(\x\) + 0(1) for all n, (where the 0(1) constant does not depend on n). 

We first show that this implies the theorem. Suppose that M\ ,M2, ... are on-line matching strategies 
for graphs G\,G2, ... It suffices to convert them to strategies M[,M 2 , ■ ■ ■ for G\,G2, ■ ■ ■ such that for 
all i, j > i strategy M 1 - is an extension of M-, i.e. on a series of requests only containing nodes from 
Gj, strategy Mj behaves exactly as M-. Because each G n finite, there are only finitely many different 
matching strategies for G n . Hence, there is a strategy M[ that equals the restriction of M n to G\ for 
infinitely many n. Therefore there is also a strategy M' 7 that is an extension of M[ and equals the 
restriction of M n to G2 infinitely often, and so on. 

It remains to show the claim. For the sake of contradiction assume that for every constant i there 
is n such that G n has not on-line matching with overhead c(\x\) + i, (and / is a list-approximator for 
c(|x|)-short programs on U). Because G„ is finite, for all n and c one can find algorithmically (using an 
exhaustive search) whether G n has on-line matching with overhead c(\x\) + i or not. One can also find 
a winning strategy for that player who wins ("Matcher" or "Requester"). Therefore for every i we can 
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algorithmically find the first n such that the graph G n has not on-line matching with overhead c(|x|) + / 
and the corresponding winning strategy for Requester for G n . 

Let that strategy play against the following "blind" strategy of Matcher. Receiving a request (x, k) 
the Matcher runs U (p) for all p € f(x), \p\ <k + c(\x\) + i, in parallel. If for some p, U (p) halts with 
the result x, he matches the first such p to x and proceeds to the next request. Otherwise the request 
remains not fulfilled. 

Consider the following machine V. On input (q,i) with \q\ = k it finds the first G n such that the 
graph G n has not on-line matching with overhead c(|x|) + i and a winning strategy for Requester, and 
runs it against the blind strategy of Matcher. Then it returns x where (x,k) is the qth request with the 
second component k. As Requester wins, there is a request (x, k) that was not fulfilled. We have 

C v (x) <C v {x) + 0{\) <k + 2\ogi + 0{\)<k + i (1) 

(the last inequality holds for all large enough i). As the request (x,k) was not fulfilled, there is no p in 
f(x) with \p\ < k + i + c(\x\). Due to CQ>, f(x) has no c(|x|)-short program for x, a contradiction. □ 

From the technical point of view, our main contributions are the following theorems. 

Theorem 2.4 (Combinatorial version of Theorem II .IK There is a computable graph with L = {0, 1}* 
with left degree D(x) = 0(|x| 2 ) which has on-line matching with overhead 0(1). 

Theorem 2.5 (Combinatorial version of Theorem 11 .2K There is a polynomial time computable graph 
with L = {0, 1}* with left degree D(x) = poly(\x\) which has on-line matching with overhead 0(log |x|). 

Theorem 2.6 (Combinatorial version of Theorem I1.3K In every graph G with L = {0,1}" that has 
off-line matching with overhead c, the maximal degree of left nodes is Q.(n 2 /(c + O(l)) 2 ). 

Theorems 12.41 12.51 and 12.61 imply Theorems 11.11 11.21 and 11.31 respectively. Moreover, our on-line 
matching strategy used in the proofs of Theorems |2.4| and |2.5| are very simple: receiving a new request 
of the form (x, k) we just find the maximal i < k + c such that there is a free hash- value of length i and 
match x with the first neighbor of x, in some order, which is not used so far. 

3 The upper bounds 

In this section we prove Theorems 12.41 and 12.51 We will need the notion of a graph with on-line 
matching, introduced in MMRSlll . 

Definition 3.1. Say that a bipartite graph has matching up to K with at most M rejections, if for any 
set of left nodes of size at most K we can drop at most M its elements so that there is a matching in 
the graph for the set of remaining nodes. A graph has an on-line matching up to K with at most M 
rejections if we can do this in on-line fashion. For M = we say that the graph has matching up to 
K. A bipartite graph is called a (K,K')-expander, if every set of K left nodes has at least K' distinct 
neighbors. 

Graphs that have matching up to K are closely related to (A',^')-expanders. Indeed, any graph 
having off-line matching up to K is obviously a (K' , ^')-expander for all K 1 < K. Conversely, by Hall's 
theorem HHal35ll any graph which is a (K' ,K') -expander for all K' <K has off-line matching up to K. 

In MMRSlll it was shown that a reduction from expanders to on-line matching is also possible. 
More specifically, every family of (2',2')-expanders, one for each i < k, sharing the same set L of left 
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nodes can be converted into a graph with the same set L of left nodes that has on-line matching up to 
2 k , at the expense of multiplying the degree by k and increasing hash- values by 1. (We will present the 
construction in the proof of Theorem 13.31 ) 

The connection between graphs with matching up to K and graphs with matching with overhead c 
is the following. If a graph G has (on-line) matching with overhead c then removing from G all left 
nodes of length different from n and all right nodes of length more than k + c(n) we obtain a graph 
with (on-line) matching up to 2 k . On the other hand, assume that for some n for all k < n we have a 
graph G n! k with L = {0, 1}" and R = {0, l} i+c (") which has (on-line) matching up to 2 k . Then the union 
G n of G n fr over all k < n has (on-line) matching with overhead c, provided all requests (x,k) satisfy 
k < \x\. At the expense of increasing the degree and c by 1, the graph G n can be easily modified to have 
(on-line) matching with overhead c unconditionally: append to all right nodes of G n and for every 
x € {0, 1}" add a new right node x\ connected to x only. 

In MMRS11I it is observed that every (l^ 1 ^ -1 ) -expander has on-line matching up to 2 k with at 
most 2 k ~ x rejections. We need a slight generalization if this fact. 

Lemma 3.2. Every (2 ,2 — 2 )-expander has on-line matching up to 2 k with at most 2 * rejec- 
tions. 

Proof. Use the following greedy strategy for on-line matching: each time a left vertex is received, 
check if it has a neighbor that was not used yet. If yes, any such neighbor is selected as the match for 
that node. Otherwise, the node is rejected. 

For the sake of contradiction, assume that the number of rejected nodes is more than 2 k - (: . Choose 
from them exactly 2 rejected nodes. By expansion property, they have at least 2 — 2 neighbors 
and all those neighbors are used by the greedy strategy (otherwise the node having a non-used neighbor 
would not be rejected). Thus we have at least 2 k — 2 k ~ (: matched left nodes and more than 2 k rejected 
nodes. Thus we have received more than 2 k requests. □ 

In particular, every (2 k , 2 k ) -expander has on-line matching up to 2 k+l with at most 2 k rejections. 
For Theorem 12.41 we will use non-explicit such graphs, for Theorem 12.51 we will need explicit such 
graphs, which we obtain from the disperser of HTSUZ071 . 

Assume that for every n and k < n we are given a (2*,2*)-expander G n>k with L = {0, 1}", R = 
{0,1}* +C W and the degree of all left nodes is at most D{n). Assume further that given n,k and a left 
node x in G n ^ we can algorifhmicaily find the list of all neighbors of x in G n ^- 

Theorem 3.3. Given a family of expanders as above we can construct a computable graph G with 
L = {0, 1}* that has on-line matching with overhead c{n) + O(logTi) and the degree of each left node 
is 0{D{n)n). The matching strategy for G is computable. Moreover, if given n,k and a left node x in 
G K) k we can find the list of all neighbors ofx in time poly(n) then G is polynomial time computable. 

Proof. The main tool is borrowed from [MRS1 1 ]: all the graphs G n ^ share the same set of left nodes 
while their sets of right nodes are disjoint. Let H„ t k denote the union of G„j over all i < k. Then H n ^ 
has on-line matching up to 2 k (without rejections). Indeed, each input left node is first given to the 
matching algorithm for G n ^-\ (that has on-line matching up to 2 k with at most 2 k ~ x rejections) and, if 
rejected is given to the matching algorithm for G n ^-2 and so on. 

Using this construction we can prove the theorem with slightly worse parameters as claimed. To 
this end identify right nodes of the graph H n ^ with strings of length k + c{n) + 1 (the number of right 

nodes of H n ^ does not exceed the sum of geometrical series 2 k+c ^ -\-2 k+c ^~ l H < 2 k+c ^ +1 ). The 

degree of H n ^ is D(n)k. 
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Recall the connection between matching up to 2 k and matching with overhead (the third paragraph 
after Definition 13. lb . We see that the family H n ^ can be converted into a graph H n with L = {0, 1}" 
and degree D(n)n(n — l)/2 + 1 having on-line matching with overhead c(n) + 2. Finally, prepend each 
right nodes of H n by a O (log n) -bit prefix code of the number n and consider the union of all H n . The 
resulting graph has on-line matching with overhead c(n) + 0(log«), its set of left nodes is {0, 1}* and 
the degree of every left node of length n is 0(D(n)n 2 ). 

Now we will explain how to reduce the degree to 0{D{ri)ri). Consider four copies of G„^_i with 
the same set L of left nodes and disjoint sets of right nodes (say append 00 to every right node to get 
the first copy, 01 to get the second copy and so on). Their union is a (2* _1 ,2 k+l ) -expander, and hence 
has matching up to 2 k+l with at most 2 k ~ l rejections. |f] Its left degree is 4D(n) and the length of right 
nodes is k + c{n) + 2. Replace in the above construction of H n the graph H n ^ by this graph. Thus the 
left degree of H n becomes 0(D(n)n) in place of 0(D(n)n 2 ). It remains to show that (the union of all 
graphs) H n has still on-line matching with overhead c(n) + O(logrc) 

Again the matching strategy is greedy. Once we receive a request (x,k) with \x\ = n, we match x 
to be if k > n. Otherwise we pass x to the matching algorithm in H n ^. If the algorithm rejects x, we 
pass x to the matching algorithm in H n ^-i and so on. We claim that we eventually find a match in one 
of the graphs H n j for i < k. To prove the claim it suffices to show that the matching algorithm for H n j 
receives at most 2 k+l input strings. This is proved by a downward induction on k (for any fixed n). For 
k = n — 1 this is obvious: we try to match in H n n -\ up to 2"~ ! strings. The induction step: by induction 
hypothesis the matching algorithm for H n ^+\ receives at most 2 k+2 input strings an thus rejects at most 
2 k of them. The matching algorithm for H n ^ thus receives at most 2 k rejected strings and at most 2 k 
new ones, coming from requests of the form (x,k). □ 

3.1 Proof of Theorem |23| 

A weaker form of Theorem 12.41 can be derived from Theorem 13.31 and the following lemma 
from HMuc02L 

Lemma 3.4. For all n and k <n, there exists a (2 k ,2 k )-expander with L = {0, 1}", R = {0, \} k+2 and 
all left nodes have degree at most n+l. 

Proof. We use the probabilistic method, and for each left node we choose its n + 1 neighbors at ran- 
dom: all neighbors of each node are selected independently among all 2 k+2 right nodes with uniform 
distribution, and the choices for different left nodes are independent too. We show that expansion prop- 
erty is satisfied with positive probability. Hence there exists at least one such graph. To estimate the 
probability that the property is not satisfied, consider a pair of sets L' and R' of left and right nodes, 
respectively, of sizes 2 k ,2 k — 1. The probability that the neighbors of all nodes in L' belong to R' is 
upper-bounded by (l/4 c )( 1 ' c )( n+c ) 2 = (l/4)(" +c ) 2 . The total probability that expansion condition is 
not satisfied, is obtained by summing over all such L',R', i.e. 

(.y"-\ 2 ,, (2M) ,- l£ (^!)%(^)^Q)%, D 

Remark 2. By the very same construction we can obtain a graph with L = {0, 1}", R = {0, 1}* +2 , 
D = n + 1 that is a (t,t) -expander for all t <2 k . Indeed, the probability that a random graph is not 
(t,t) -expander is at most Q)' (we may replace 2 k by t in the above formulas). By union bound, the 

5 One can also consider the union of G„^._i and G n t, which also has matching up to 2 i+I with at most 2 t_1 rejections. 
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probability that it happens for some t < 2 k is at most the sum of geometric series £ 2 =1 {Vf < 1- By 
Hall's theorem, this graph has off-line matching up to 2 k . An interesting open question is whether there 
is a graph with the same parameters, i.e. L = {0, 1}", R = {0, lj^+^W, D = 0(n), that has on-line 
matching up to 2 k . 

From this lemma and Theorem 13.31 we obtain a computable graph with on-line matching with 
overhead O(logrc), degree 0(|*| 2 ) and L = {0, 1}* We now need to replace the 0(log«) overhead by 
0(1). Recall the O(logrc) appeared from the prefix code of n added to hash values. To get rid of it 
we need a computable graph Fk in place of previously used G n x with the same parameters but with 
L = {0, l} >k , and not L = {0, 1}". Such a graph is constructed in the following 

Lemma 3.5. For every k there is a computable bipartite graph Fk with L = {0, 1}-* R = {0, l} k+3 that 
is a (2 k ,2 k ) -expander and the degree of every left node x is 0{\x\). 

Proof. We first build such a graph with left nodes being all strings of length between k and K = 2 k+3 . 
This is again done by probabilistic method: we choose \x\ +3 neighbors of every node x independently. 
Let Li stand for all left nodes of length i. For any i G k,...,K, the probability that all elements of a 
fixed L' C are mapped to a fixed set of size at most 2 k — 1 at the right is at most (^■)^' +3 ^' i . The 
probability that some tj elements in L, are mapped into a fixed set of 2 k — 1 elements at the right is 
bounded by 

») =(?) y s y £ y =u) u 

If Hf=k t i = t w i tn ? = 2*, the probability that the union of neighbors of t\ elements in L^, tk+\ elements 
in Lk+u - - - , and tx elements in Lk are mapped to a fixed set of size at most 2 k — 1 is bounded by 
Yli(j^) 2 '' = {jk) 2 '- Multiplying by the number K 2k ~ l < K' of different right sets of size K— 1, and 
multiplying by the number K' of different solutions to the equation Yf=k *i = ^> we ^nd 

Hence, the total probability to randomly generate a graph that is not an expander is strictly less than 1. 
Therefore, a graph satisfying the conditions must exist, and can be found by exhaustive search. 

On the left side, we now need to add the strings of length larger than K = 2 k+3 . These nodes are 
connected to all the nodes on the right side. Thus the degree of every such node x is 2 k < \x\/2 3 < 0(|oc|) 
and we are done. □ 

Remark 3. By the very same construction we can obtain a graph with L = {0,1}-*, R = {0, 1}* +3 , 
D = 0(n) that is a (t,t) -expander for all t < 2 k and thus has off-line matching up to 2 k (use the union 
bound over all f e {it,... ,K}). An interesting open question is whether there is a graph with the same 
parameters that has on-line matching up to 2 k . 

Now we can finish the proof of Theorem 12.41 Appending all 2-bit strings to all the right nodes of 
the graph Fk-\ (and thus increasing the degree 4 times) we obtain a (2* _1 ,2* +1 ) -expander Hk- The 
union of Hk over all k is a computable graph, whose left degree is C?(|^| 2 ), and the set of left nodes is 
{0, 1}*. It has on-line matching with constant overhead. This is proved by the downward induction, as 
in Theorem 13 .3 1 Indeed, by step s in the matching, there are only finitely many requests for matching. 
By a downward induction on k we can again prove that the number of matching requests in is at 
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most 2 k+l . Now the base of induction is the maximal k for which there has been at least one request 
for matching in up to step s. We conclude that the request made at step s is satisfied and, since this 
holds for every s, we are done. 



3.2 Proof of Theorem |23] 

By Theorem l3.3l we have to construct for every k < n an explicit (2 ,2 ) -expander of left degree poly (n), 
with 2" left nodes and po\y(n)2 k right nodes. A graph is explicit if there is an algorithm that on input 
x € {0, 1}" = L lists in poly(«) time all the neighbors of x. 

The proof relies on the explicit disperser graphs of Ta-Shma, Umans, and Zuckerman from Theo- 
rem [3?7]below. 

Definition 3.6. A bipartite graph G = (L,R,E) is a (K, 8)-disperser, if every subset B C L with \B\ > K 
has at least (1 — 8)\R\ distinct neighbors. 

Theorem 3.7. [Ta-Shma, Umans, Zuckerman HTSUZ07\l l For every K,n and constant 8, there exists 
explicit (K, 8)-dispersers G = (L = {0, l} n ,R = {0, l} m ,E CLxS) in which every node in L has 
degree D = poly{n) and \R\ = ^-3—, for some constant a. 

Given n and k we apply this theorem to K = 2 k and 5 = 1/2. We obtain a (2 k , ^f-) -expander with 

degree D = poly(?i), L = {0, 1}" and \R\ = ^p-- Consider t = max{l, [§75]} disjoint copies of this 
graph and identify left nodes of the resulting graphs (keeping their sets of right nodes disjoint). We get 
an explicit (2 , 2 k ) -expander with 2" left and 2^'poly(?i) right nodes and degree poly (n)t = poly(«). 

4 The lower bounds 
4. 1 Proof of Theorem 12^1 

Assume that G has off-line matching with overhead c. Let denote the induced graph that is 

obtained from G by removing all right nodes of length more than k or less than £. The graph G[0,& + c] 
is obviously a (2 k , 2 k ) -expander for every k. As there are less than 2 k ~ l strings of length less than k—1, 
it follows that the graph G[k — l,k + c] is a (2 k ,2 k ~ l + 1) -expander. 

The next lemma inspired by HKST541 (see MRTSOOi Theorem 1.5]) shows that any such expander 
must have large degree. 

Lemma 4.1. Assume that a bipartite graph with 2 £ left nodes and 2 k+c right nodes is a {2 k ,2 k - 1 + iy 
expander. Then there is a left node in the graph with degree more than D = min{2 <:_2 , (£ — k)/(c + 2)}. 

Proof. For the sake of contradiction assume that all left nodes have degree at most D (and w.l.o.g. we 
may assume that all degrees are exactly D). We need to find a set of right nodes B of size 2 k ~ l and 
2 k left nodes all of whose neighbors lie in B. The set B is constructed via a probabilistic construction. 
Namely, choose B at random (all ( 2 *-i ) sets nave equal probabilities). The probability that all neighbors 
of a fixed left node are in B is equal to 



(2 k+c -D\ 
\2 k -i-D) 



2 *-i(2*- 1 -l)...(2*- 1 -Z) + l) 
2 k +C (2k+c _ 1) . . . (tf+c -D + 1) ' 
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Both products in the numerator and denominator have D factors and the ratio of corresponding factors 
is at least 



2 k + c -D+ \ ~ 

(the last inequality is due to the assumption D < 2 k ~ 2 ). Thus the probability that all neighbors of a fixed 
left node are in B is at least 2~ D( - c+2 \ Hence the average number of left nodes having this property is at 
least 2^~ D ( C+2 ) , which is greater than or equal to 2 k by the choice of D. Hence there is B that includes 
neighborhoods of at least 2 k left nodes, a contradiction. □ 

This lemma states that at least one left node has large degree. However, it implies more: if the 
number of left nodes is much larger than 2 , then almost all left nodes must have large degree. Indeed, 
assume that a bipartite graph with 2 k+c right nodes is a (2*,2* _1 + l)-expander. Choose 2 e left nodes 
with smallest degree and apply the lemma to the resulting induced graph (which is also a (2 k ,2 k ~ l + 1)- 
expander). By the lemma, in the original graph all except for less than 2 l nodes have degree more than 
D = min^- 2 , (£ -k)/(c + 2)}. 

Choose n/4 < k < n/2. As noticed, the graph G[k— \ ,k+c] is a (2 k ,2 k ~ l + 1) -expander and has 
less than 2 k+c+l right nodes. By Lemma [4j] (applied to £ = 3n/4, k, c + 1), all except for at most 2 3n / 4 
left nodes of G[k— \,k + c] have degree at least n/4(c + 3). 

Pick now £ different &'s that are c + 2 apart of each other, where £ is about n/{4{c + 2)). For most 
left nodes for all picked k there are n/4(c + 3) edges from those nodes in G[k — 1 , k + c] . As all picked 
k's are c + 2 apart of each other, the degree of all those nodes is Q.{n 2 / (c + 3) ). 

4.2 Proof of Theorem H3 

The size of list-approximators is closely related to total conditional Kolmogorov complexity, which 
was first introduced by A. Muchnik and was used in HVer09l IBaulOi Total conditional Kolmogorov 
complexity with respect to U is defined as: 

CT v (u |v) =min{ \q\ : U(q,v) = u/\Vz[U(q,z) I}} , 

where U (q,z) I means that U (q,z) halts. If U is a standard machine then CTu{u\v) < CTy(u\v) + cy 
for every machine V. The connection to list-approximators is the following: 

Lemma 4.2. If f is computable function that maps every string to a finite list of strings then 
CTu(p\x) < log|/(x)| +0(1) for any standard machine U and every p in f(x). The constant in 
O-notation depends on f and U. 

Proof. Let V(j,x) stand for the y'fh entry of the list f(x), if j < \f(x)\, and for the empty string (say) 
otherwise. Obviously CTy(p\x) < log \f(x)\ for all p in f(x). Hence CTjj{p\x) < log \f(x)\ + 0(1). 

□ 

Thus to prove the theorem it suffices to construct a standard machine Uq such that for infinitely 
many x every O-short p for x with respect to Uq satisfies CTu(p\x) > \x\ — 0(1). To this end we fix 
any standard machine U and construct another machine V such that for some constant d and for every 
integer k there are strings p,x such that: 

(a) p is the unique O-short program for x with respect to V, 

(b) Cu{x)>k, 
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(c) \x\ = \p\ =k + d, 

(d) CTu(Op\x) >k. 

Once such V has been constructed, we let Uo(Oq\z) = V(q\z) and Uo{\ d+2 q\z) = U (q\z). The latter 
equality guarantees that Uq is a standard machine. And both equalities together with items (a), (b) and 
(c) imply that Op is the unique 0-short program for x with respect to Uq. Finally, item (d) guarantees 
that its total complexity conditional to x is at least |x| — d — 1. 

The construction of V can be described in game terms. (The game-based technique in recursion 
theory was introduced by Lachlan HLac701 and further developed by A.Muchnik and others [ MMSV10I 
IVer08llMSVT2l .) 

Description of the game. The game has integer parameters k,d and is played on a rectangular grid 
with 2 k+d rows and 2 k+d columns. The rows and columns are identified with strings of length k + d. 
Two players, Black and White, play in turn. In her turn White can either pass or put a pawn on the 
board. White can place at most one pawn in each row and at most one pawn in each column. Once a 
pawn is placed, it can not be moved nor removed. In his turn Black can either pass, or choose a column 
and disable all its cells, or choose at most one cell in every column and disable all of them. If a player 
does not pass, we say that she/he makes a move. Black is allowed to make less that 2 k+l moves. The 
game is played for an infinite time and White looses if at some point after her turn, all her pawns are in 
disabled cells. 

We will show that, for d = 3, for every k, White wins this game. More specifically, there is a 
winning strategy for White that is uniformly computable given k. Assume that this is done. Then 
consider the following "blind" strategy for Black: start enumeration of all strings x with Cjj{x) < k 
and all strings q of length less than k such that U (q,x) J, for all x of length k + d. That enumeration 
can be done uniformly in k. In his rth turn Black: disables all cells in the xth column, if on step 
t in this enumeration a new x of length k + d with C\j{x) < k appears; disables all cells (p,x) with 
\ x \ = \p\ = k + d, U(q,x) = Op, if on step t a new string q of length less than k appears such that 
U (q,x) I for all x of length k + d; and passes if none of these events occurs. Note that the total number 
of Black's moves is less that 2 k + 2 k = 2 k+l , as required. 

Now consider the following machine V(p): let k = \p\ — d and let the White's computable winning 
strategy play against Black's blind strategy. Watch the play waiting until White places a pawn on a 
cell (p,x) in pth row. Then output x and halt. Note that such x is unique (if exists), as White places at 
most one pawn in each row. And that cell (p,x) satisfies all the requirements (a)-(d). Thus it suffices 
to design a computable winning strategy for White. 

A winning White's strategy. The strategy is a greedy one. In the first round White places a pawn 
in any cell. Then she waits until that cell becomes disabled. Then she places the second pawn in 
any enabled cell that lies in another row and another column and again waits until that cell becomes 
disabled. At any time she chooses any enabled cell that lies in a row and a column that both are free of 
pawns. In order to show that White wins, we just need to prove that there is such cell. Indeed, Black 
makes less than 2 k+l moves, thus White makes at most 2 k+l moves. On each of Black's moves at most 
2 k+d cells become disabled. On each of White's moves at most 2 k+d+l cells becomes non-free because 
either their column or row already has a pawn. Thus if the total number of cells is more than 

r^k+X^+d _|_ jk+\^k+d+\ _ g _ 2 2k 2 d 

we are done. The total number of cells is 2 k+d 2 k+d = 2 2k 2 2d . As 2 2d grows faster that 6 • 2 d , for large 
enough d (actually for d = 3) the total number of cells is larger than the number of disabled or non-free 
cells. The theorem is proved. 
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Note that changing the construction a little bit we can prove the same statement for c-short pro- 
grams for every c. To this end we just need to let Uo(l c+d+2 q) = U (q) instead of Uo(\ d+2 q) = U (q). 
The optimal machine Uq constructed in this way depends on the choice of c, which is inevitable by 
Theorem ll.il 

5 Other applications of explicit graphs with on-line matching 

The two first applications are related with the resource bounded Kolmogorov complexity. Recall a 
machine U is called standard if for any machine V there is a total computable function / such that 
U (t(p),z) = V(p,z) and \t(p) \ < \p\ + 0(1) for all p,z. In this section we assume that t is polynomial 
time computable and that running time of U (t(p),z) is bounded by a polynomial of the computation 
time of V (p,z). By Cjj(x\z) we denote the minimal length of p such that U (p,z) = x in at most T steps. 

Muchnik's Theorem IMuc02[|MRSll|| . Let a and b be strings such that \a\ = n and C(a \ b)=k. 
Then there exists a string p such that (1) \p\ = k + 0(logn), (2) C(p \ a) = 0(logn), (3) C(a \ p,b) = 
O(logn). 

In our improved version, we replace (2) by (2') C q ^ n \p \ a) = O(logn), where q is a polynomial. 

Proof. Fix an explicit graph with L = {0,1}*, polynomial degree, and that has computable on-line 
matching with logarithmic overhead. Given a string b run the optimal machine U(q,b) in parallel 
for all q and once for some q, U (q,b) halts with the result x pass the request (jc, \q\) to the matching 
algorithm in the graph. It will return a neighbor p of length at most \q\ + 0(logn) of x. At some 
moment a shortest program q for a conditional to b will halt and we get the sought p. 

As the graph is explicit and has polynomial degree, we have C poly ( n \p \ a) = 0(logn) (requirement 
(2)). Requirement (1) holds by construction. Finally, C(a \ p,b) = 0(\ogn) as given p and b we may 
identify a by running the above algorithmic process (it is important that a is the unique string that was 
matched to p). □ 

Distinguishing complexity HBFL01L 

Let V be a machine, x a string and T a natural number. The distinguishing complexity CDy (x) with 
respect to V is defined as the minimal length of p such that U(p,x) = 1 (p "accepts" x) in at most 
T steps, and U(p,x') = for all x' ^ x (p "rejects" all other strings). From our assumption for the 
standard machine U it follows that for every machine V there is a polynomial / and a constant c such 
that Cd(j {T) (x) < CDl{x)+c. Indeed, let p is a shortest distinguishing program for x working in T 
steps with respect to V. Then t(p) is a program for U that accepts x in poly(r) steps and rejects all 
other strings. 

For a set A of binary strings let A =n stand for the set of all strings of length n in A. 

Theorem 5.1 ( MBFLOll ). For every function e(n) (mapping natural numbers to numbers of the form 
1 /natural) computable in time poly(n) there is a polynomial f such that for every set A, for all x € A =n 
except for a fraction e(n), CD^"' ,A (x) < log|A = "| +polylog(n/e(n)). 

We mean here that the set A is given to the standard machine U as an oracle (so we assume that the 
standard machine is an oracle machine and all the requirements hold for every oracle.) 
In our improved version, we obtain CD^ n ^(x) < log |A="| + 0(log(n/e(n))). 

Proof. For our improvement we need for every n, k<n and £ a bipartite graph G n ^z with L = {0, 1}", 
R = {0, i} k +°( lo & n / e ) an d degree poly(n/e) that has the following property: 
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for every subset S of at most 2 k left nodes for every node x in S except for a fraction e there 
is a right neighbor p of x such that p has no other neighbors in S. 

Assume that we have such an explicit family of graphs G, u u,e- Explicit means that given n, k, e, a 
left node x and i we can in polynomial time find the zth neighbor of x in G n ^,e ■ Then we can construct 
a machine V that given a tuple (p, i, n,k), a string x and A as oracle verifies that x is in A =n and that x is 
the z'-th neighbor of p in Gu^w. If this is the case it accepts and rejects otherwise. By the property 
of the graph, applied to S = A =n and k = [log |S|] we see that 

CD f v {n)A (x) < \(p,i,n,*)\ < log |A="|+0(logn/e(n)) 

for some polynomial f(n) for all but a fraction e(n) for x € A = ". By the assumptions on U the same 
inequality holds for U. 

The graph G n ^,e is again obtained from the disperser of HTSUZ071 . Given n, k and e we apply 
Theorem |3.7| to K = 2 k e and 5 = 1/2. We obtain a (K, -expander with degree D = poly(«), 

L = {0, 1}" and \R\ = ^p-- Consider t = max{l, disjoint copies of this graph and identify left 

nodes of the resulting graphs (keeping their sets of right nodes disjoint). We get an explicit (2 k e,2 k e)- 
expander with 2" left and 2 /: poly(«)/£ right nodes and degree D = poly (n)t/e = poly («)/£. 

This graph, called H n ^ £ , has the following "low-congestion property": for every set of2 k left nodes 
S for every node x in S except for a fraction e there is a right neighbor p of x such that p has at most 
D/e neighbors in S. 

Indeed, the total number of edges in the graph originating in S is at most \S\D. Thus less than 
\S\D/(D/e) = \S\e right nodes are "fat" in the sense that they have more than D/e neighbors landing in 
S. By the expander property of H n ^, e there are less than e\S\ left nodes in S that have only fat neighbors. 

It remains to "split" right nodes of tL n ,k,e. so that D/e becomes 1 . This is done exactly as in MBFL011 . 
Using the Prime Number Theorem, it is not hard to show (Lemma 3 in MBFL0110 that for every set W 
of d strings of length n the following holds: for every x G W there is a prime number q < Adn 2 such 
that x^x' (mod q) for all i'£W different from x (we identify here natural numbers and their binary 
expansions). 

We apply this lemma to d = D/e. To every right node p in H n ±^ we add a prefix code of two 
natural numbers a,q, both at most Adn 2 , and connect a left node x to (p,a,q) if x is connected to p 
in H n fc e and x = a (mod q). We obtain the graph G nj k,e we were looking for. Indeed, for every S 
of 2 k left nodes for all x € S but a fraction of e there is a neighbor p of x in H n ^ e that has at most 
d = D/e = poly(«)/£ neighbors in S. Besides there is a prime q < 4n 2 d = poly(«)/e such that x^x' 
(mod q) for all neighbors x' of p different from x. Thus the neighbor (p,q,x mod q) of x in G n ^ e has 
no other neighbors in S. 

The degree of G n ^ e is D x (An 2 D/e) 2 = poly(«)/e 2 . The number of right nodes is 

(poly \n)2 k /e)(An 2 D/e) 2 = 2*poty '(«) / ■ 
Thus right nodes can be identified with strings of length k + 0(logn/e) and we are done. □ 
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